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Significance for public health 

While the benefits of collaboration have become widely accepted, and the 
practice of collaboration is growing within the public health system, the abil- 
ity to measure, document, and strategize to affect practice has been weak. 
However, the need for evaluation and analysis in these areas is strong 
because collaboration has the potential to improve the processes of health- 
care which can create better outcomes, but also reduce the cost of delivering 
services by eliminating waste, unnecessary work, and rework. In the case of 
collaboration, it is important to recognize that both the resources (inputs) 
and activities carried out (processes) must be addressed together to ensure or 
improve the quality of care. Once these dimensions are addressed, then prac- 
tice and policy can be affected through strategic planning, involving the 
workforce that makes up the bulk of leadership within public health collabo- 
ratives (PHCs). 



Abstract 

While the benefits of collaboration have become widely accepted 
and the practice of collaboration is growing within the public health 
system, a paucity of research exists that examines factors and mecha- 
nisms related to effective collaboration between public health and 
their partner organizations. The purpose of this paper is to address 
this gap by exploring the structural and organizational characteristics 
of public health collaboratives. Design and Methods. Using both social 
network analysis and traditional statistical methods, we conduct an 
exploratory secondary data analysis of 11 public health collaboratives 
chosen from across the United States. All collaboratives are part of the 
PARTNER (www.partnertool.net) database. We analyze data to identi- 
fy relational patterns by exploring the structure (the way that organi- 
zations connect and exchange relationships), in relation to percep- 
tions of value and trust, explanations for varying reports of success, 
and factors related to outcomes. We describe the characteristics of the 
collaboratives, types of resource contributions, outcomes of the collab- 
oratives, perceptions of success, and reasons for success. We found 
high variation and significant differences within and between these 
collaboratives including perceptions of success. There were signifi- 
cant relationships among various factors such as resource contribu- 
tions, reasons cited for success, and trust and value perceived by 
organizations. We find that although the unique structure of each col- 
laborative makes it challenging to identify a specific set of factors to 
determine when a collaborative will be successful, the organizational 
characteristics and interorganizational dynamics do appear to impact 
outcomes. We recommend a quality improvement process that sug- 
gests matching assessment to goals and developing action steps for 
performance improvement. 



Introduction 

In the United States, the Ten Essential Services of Public Health 
were designed in 1994 as a framework for the National Public Health 
Performance Standards Program of the Center for Disease Control and 
Prevention.^ Notably, they included the goal of mobilizing community 
partnerships and action to identify and solve health problems. At the 
local level, this meant clearly stating the importance of identifying rel- 
evant stakeholders and involving them in the process, building coali- 
tions to strengthen human and material resources, and facilitating 
partnerships among groups and associations, some who may be atypi- 
cal partners {e.g. organizations outside of public health, for example 
public safety, business, and schools).^-^ With these guiding responsi- 
bilities, public health departments are working toward strengthening 
their network of partners through interorganizational networks in 
order to better coordinate health services and resources leading not 
only to less duplication but more substantially, increased cohesion and 
improved adoption of interventions and programme implementation."^ 

More recently, collaboration across sectors continues to be formal- 
ized. For example, the Public Health Accreditation Board (PHAB), the 
governance body that provides guidance to Public Health Departments 
(PHDs) in preparation to meet the evolving set of standards and meas- 
ures for voluntary accreditation, published the revised Proposed 
State/Local Standards and Measures documents,^ emphasizing collabo- 
ration as a core competency. The fourth domain addressed in these 
standards is Engage the Public Health System and the Community in 
Identifying and Addressing Health Problems. They emphasize taking 
inventory of partnerships and demonstrating their relevance to 
demonstrate improved capacity. 

The increased practice of collaboration echoes what is emphasized 
in the literature. For example, relations that involve the exchange of 
resources and knowledge among sectors are the norm and certainly 
the latest trend in successful social service models.^"^^ Networks of 
community organizations working for a common purpose are thought 
to increase community capacity to meet social needs.^^-^^ The benefits 
of interorganizational community networks include: i) bringing 
diverse groups and resources together; and ii) addressing issues that 
no group can resolve by itself. Blau and Rabrenovic^"^ found that 
interorganizational linkages are more important than bureaucratic 
hierarchies for controlling and coordinating work. . . linkages are used 
to integrate programs within a community, coordinate client services, 
obtain resources, and deal with governmental agencies (p. 328). 
However, despite these published benefits of collaboration, it is impor- 
tant to note that in addition to the anticipated benefits, collaborations 
are often characterized by a high-level of dissatisfaction with their actu- 
al outcomes relative to expectations, and correspondingly, a high rate of 
failure (p. 326).^^ This is in large part due to the challenges of collab- 
orating such as transaction costs, potential conflict, and lack of 
accountability. 

This mix of accountability, evidence in research, and experience in 



[page 170] 



[Journal of Public Health Research 2012; l:e27] 



Oi»EN/7l ACCESS 



Article 



practice has in turn led to the rise of public health collaboratives (also 
referred to as coalitions) as a strategy for addressing the most pressing 
public health problems in communities. This paper is an exploratory 
analysis of eleven public health collaboratives. 

While the benefits of collaboration have become widely accepted,^^-^^ 
and the practice of collaboration is growing within the public health 
system, the ability to measure, document, and strategize to affect prac- 
tice has been weak.^^-^^ However, the need for evaluation and analysis 
in these areas is strong because collaboration has the potential to 
improve the processes of healthcare which can create better outcomes, 
but also reduce the cost of delivering services by eliminating waste, 
unnecessary work, and reworkP In the case of collaboration, it is 
important to recognize that both the resources (inputs) and activities 
carried out (processes) must be addressed together to ensure or improve 
the quality of care ?^ Once these dimensions are addressed, then prac- 
tice and policy can be affected through strategic planning, involving the 
workforce that makes up the bulk of leadership within public health col- 
laboratives (PHCs). 

Research questions 

Analysis to understand and explain the process of collaboration 
requires an understanding of the relationships among a number of fac- 
tors (e.g. in a dynamic system with nested levels of interaction.^^-^^ For 
example. Mays and Scutchfield^ note theit Public health activities in the 
United States are delivered through multiple public and private organi- 
zations that vary widely in their resources, missions, and 
operations.. .without strong coordination mechanisms, these delivery 
arrangements may perpetuate large gaps, inequities, and inefficiencies 
in public health activities (p. 1). The diversity of these multiple organ- 
izations and the varying context that each community contributes to 
the functioning of these collaboratives creates a level of complexity 
within public health systems that cannot be evaluated with a single 
case study or anecdotal analysis.^^-^^ Rather, analysis across a wider 
range of collaboratives is required to identify determinants of effective- 
ness in public health collaboration. While the practice of collaboration 
in public health is on the rise and accepted as an essential practice in 
the field, the study of collaboration has not kept up. Understanding the 
process of collaboration, differences among practice, and factors of suc- 
cess is important to ensure that collaboration is conducted with evi- 
dence to back up the process. In this paper, we conduct an exploratory 
analysis of 1 1 public health collaboratives across the United States. We 
analyze network data to identify relational patterns by exploring the 
network structure (the way that organizations connect and exchange 
relationships). We then take a closer look at the relationships between 
members in these collaboratives to determine how certain organiza- 
tional characteristics are associated with outcomes of these relation- 
ships such as how frequently partners interact and their perceptions of 
partners' trust and value to the network. Specifically, we address the 
following two research questions: 

RQl: What are network characteristics of public health collaboratives? 

RQ2: What are the relationships between network members' character- 
istics and quality of connections (e.g. trust, value, and frequency 
of interactions) among members of collaboratives? 



Design and Methods 

Data collection 

This study was an exploratory, secondary data analysis of 11 public 
health collaboratives chosen from the PARTNER (Program to Analyze, 
Record, and Track Networks to Enhance Relationships - www.partner- 
tool.net) database. PARTNER is a social network analysis tool used by 



public health agencies across the country to assess and evaluate their 
efforts to collaborate with other organizations/agencies across the pub- 
lic health system. These efforts include but are not limited to partner- 
ships for shared programming and/or funding, implementation of inter- 
ventions and programmes, knowledge sharing, and community capaci- 
ty building with a diverse range of other organizations/agencies. At the 
heart of these collaborations is the exchange of resources such as in- 
kind service, health expertise and non-health expertise, funding, data, 
education, and programmatic services shared to accomplish mutual 
goals related to public health practice. PARTNER is an online survey 
and analysis tool designed to measure and monitor collaboration 
among people and/or organizations. The online PARTNER survey col- 
lects data on individual organizational characteristics such as type of 
organization, length of time participating in the partnership, what 
resource contributions the organization provides to the collaborative, 
what outcomes the collaborative focuses on, the perceived level of suc- 
cess for accomplishing collaborative goals, and reasons for successful 
collaborations. The survey also contains relational questions that ask 
organizations to identify other organizations in the collaborative that 
they work with and then asks subsequent questions about how fre- 
quently they interact with that organization, the quality of those inter- 
actions, and perceptions of trust and organizational value for each part- 
ner organization (using three measures of trust and three measures of 
value).^'^ These survey items are listed in Table 1. 

This project was reviewed by the Colorado Multiple Institution 
Internal Review Board and was deemed exempt and approved for analy- 
sis as a secondary data set. 



Table 1. Partner survey questions. 



Demographic questions (organizational description) 



Job title, length of time as a member of the collaborative, types of activities 
engaged in, outcomes of the collaborative, resources contributed to the col- 
laborative. 



Perception of success 



How successful has your collaborative been at reaching its goals? 

Not Successful, Somewhat Successful, Successful, Very Successful, Completely 

Successful 

Collaborative outcomes; factors contributing to successful outcomes. 



Relational questions 



Please list all organizations/ divisions/ agencies/ programs with whom you 
have a relationship with to meet the goals of your collaborative. 
Once each respondent selected their organizational partners, they were 
asked to answer the following questions: 

1) Frequency of interactions with partner 

None, Once a Year, Every Few Months, Every Month, Every Few Weeks, Once 
a Week, Every Day 

2) Level of quality of activity in the relationship 
None, Coordinated, Cooperative, Integrated 

3) Extent of value as: 

a) power/influence; 

b) level of involvement; 

c) resource contribution 

None, A Small Amount, A Fair Amount, A Great Deal 

4) Extent of trust as: 

a) reliable; 

b) in support of the mission; 

c) open to discussion 

None, A Small Amount, A Fair Amount, A Great Deal 
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Sample 

The 11 coUaboratives used for this study are a purposeful sample, 
selected from the PARTNER database for their common characteristics. 
CoUaboratives that were most comparable along a set of criteria were 
selected. These selection criteria were based on the following: i) mis- 
sion (we chose coUaboratives that were focused on addressing a set of 
identified public health needs in their communities (for example, oral 
health, early childhood intervention, maternal health, health dispari- 
ties, and general community health); ii) organizations as the unit of 
analysis in their data (no individuals were represented in the data); 
iii) use of the same PARTNER survey questions (or questions that were 
easily coded to match the PARTNER survey questions); iv) State or local 
public health department as members of the collaborative. Less than 
half of these coUaboratives had some kind of funding. All coUaboratives 
opted to participate in the PARTNER survey. The 1 1 public health col- 
laboratives ranged in size from having 16 to 433 member organizations 
totaling 868 organizations. Of those, 262 responded to the surveys 
which resulted in 3738 dyadic ties between organizations who were 
members of the coUaboratives. Data were collected between July 2010 
and February 2011 in the United States. Response rates for each net- 
work dataset (11 in total) ranged from 45% to 100%, with a mean 
response rate of 76%. While we analyzed each data set at the whole net- 
work (collaborative) level in our descriptive social network analysis, we 
used a different approach in our inferential analysis to account for the 
variation in response rates. Specifically, because some of the response 
rates were low enough to cause uncertainty of whether a full represen- 
tation of the network was attained,^^ we opted to aggregate the data 
into one dataset consisting of attribute data for the respondent organ- 
izations and reported dyadic relationships. Each dyad (N=3738) con- 
sisted of two organizations and the measure of interaction between 
them. In this way, we are able to take a closer look at the way that 
organizations work together with partners they identify as having a 
relationship with, rather than comparing across the 1 1 coUaboratives. 

Analysis 

To address the research questions posed above, we analyzed the data 
at three levels (Figure 1). First, we described some of the organization- 
al level characteristics of members of public health coUaboratives. Then, 
we describe the 11 coUaboratives at the whole network level in terms of 




LEVEL 



Figure 1. Three levels of analysis (Whole Network, Organiza- 
tional, Dyad). 



their network characteristics using Social Network Analysis (SNA)^'^ 
and descriptive statistics. SNA is a methodology used to gather and ana- 
lyze data to explain the degree to which network actors connect to one 
another and the structural makeup of coUaborative relationships.^^ We 
present network maps of the 11 coUaboratives and report on whole net- 
work characteristics including reciprocity, centralization, and transitiv- 
ity.^^ Finally, we examined the data at the dyad level (to analyze relation- 
ships between each pair of organizations) using Hierarchical Linear 
Modeling (HLM) and regressions to examine the relationships between 
network members' characteristics (organization type, members' percep- 
tions of outcome achieved, and members' perception for factors con- 
tributing to success) and quality of connections (trust, value, and fre- 
quency of interactions). We used HLM to account for the nested features 
of these data (dyads nested within organizations), specifically to exam- 
ine how variables measured at one level affect relationships at anoth- 
erP HLM accounts for the dependent nature of nested data using multi- 
level models to estimate variation on one-on-one organizational rela- 
tionship (dyad level) outcomes using a restricted maximum likelihood 
approach. Using this method, the interactions that occur across levels 
are taken into account and can be estimated. 



Results 

We used SNA and descriptive statistics to answer research Question 
1, What are network characteristics ofPH coUaboratives? 

Organizational characteristics of members of the 
eleven coUaboratives 

Organizations represented in these coUaboratives include public 
health, businesses, dental services, educational institutions, faith- 
based organizations, government organizations, funders, hospitals, law 
enforcement, non-profit community and advocacy organizations, pri- 
mary care, health clinics, hospices, professional organizations, health 
insurance organizations, among others. The role and number of public 
health departments and specifically how they were embedded in each 
coUaborative structure varied among the 11 groups. The survey asked 
respondents to rate these organizational partners in terms of levels of 



■ Total value mean ■ Total trust mean 




Figure 2. Organizational rankings on trust and value. 
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trust and value to the collaborative. Overall, funders, primary 
care/health clinics, hospitals, and public health were rated highest on 
these two measures by respondents (Figure 2). While public health 
organizations do not rank highest on value, they have the highest rank- 
ing for trust (they were rated as most trusted by their partners). 
Moreover, independent sample ^-tests indicate there is a significant 
difference between public health and non-public health organizations 
(total trust, t=3.538, P=0.001). 

When asked what types of resources each respondent organization 
(N=262) contributes to their public health collaborative, the following 
were reported: community connections (n=189), specific health 
expertise (n=171), information/feedback (n=166), volunteers 
(n=116), facilitation (n=106), data sets (n=94), expertise other than 
health expertise (n=84), paid staff (n=82), in-kind resources (n=74), 
decision making (n=50), and funding (n=26). Respondents also 
reported which outcomes were achieved due to the work of the collab- 
orative. The most commonly mentioned outcome was public aware- 
ness, followed by increased knowledge sharing, increased resource 
sharing, community support, increased services, reduction of health 
disparities, new sources of data, new policies, and finally education. Of 
these outcomes reported, respondents identified the most important 
outcomes as increased services to clients (n=66, 24.9%). 

Whole network analysis to describe the network char- 
acteristics of the eleven collaboratives 

Figure 3 presents network images of all eleven collaboratives to 
demonstrate the variation in how these types of networks are struc- 
tured. In these images, nodes are connected when a formal working 
relationship exists between the organizations and interaction occurs at 
least once a year. All collaboratives had unique network structures 
(Figure 3), meaning some were very dense (they had many relation- 
ships between the partners) (collaborative 10), while others were less 
dense (collaborative 9), some included public health organizations 
(indicated as large nodes) as the centralized members of the network 
(collaborative 4), others were more clustered by these sector differ- 
ences (collaborative 10), and still others had structures that consisted 
of several distinct components (collaborative 4). No two collaboratives 
had very similar structures. 

We used three social network analysis metrics to describe the net- 
work characteristics of these collaboratives. These included reciproci- 
ty, transitivity, and centralization. Reciprocity is a measure that tells us 
what percentage of relationships are reciprocated between organiza- 
tions. Transitivity is a measure that tells us about the level of informa- 
tion flow in a collaborative. Centralization is a measure that indicates 
how many organizations play central positions (the lower the number 
the fewer the number of organizations that are central).^^ Across the 11 
collaboratives we saw varying levels of all three of these scores. 
However, we did identify a pattern (Figure 4) that indicates that when 
reciprocity is high or low within a collaborative, transitivity mimics that 
behaviour. In other words, in these 1 1 collaboratives, when there is a 
higher instance of reciprocal relationships in a network there is also a 
tendency to see greater information flow. On the other hand, when rec- 
iprocity is low, information flow (transitivity) is also low. Centralization 
did not seem to follow any kind of similar pattern among these collab- 
oratives. We also compared the collaboratives at the whole network 
level in terms of how the respondents reported levels of success. 
Respondents were asked to answer the question How successful has 
your collaborative been at reaching its goals? While the top responses 
included successful (37% of respondents) and somewhat successful 
(36% of respondents), the variability of these responses within collab- 
oratives was high (20% reported their collaboratives as being very suc- 
cessful, 4% reported completely successful and 3% reported not success- 



ful). The responses lacked consensus from respondents in any one col- 
laborative to demonstrate agreement about whether their collabora- 
tives are successful and to what degree. 
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Figure 3. Comparison of three network maps (large orange nodes 
indicate Public Health Organizations). 



— Reciprocity — ^ Transitivity — Centralization 
0.60t 




Figure 4. Comparison across the 1 1 collaboratives in terms of rec- 
iprocity, transitivity, and centralization. 



Of EH p\ ACCESS 



[Journal of Public Health Research 2012; l:e27] 



[page 173] 



Varda and Retrum 



Dyad level analysis 

HLM was used at the dyadic level to answer research Question 2: What 
are the relationships between network members' characteristics and qual- 
ity of connections (trust, value, and frequency of interactions) among 
members of collaborativesl We tested three different types of HLM mod- 
els to explore whether or not certain organizational characteristics 
(including an organization's type, their perceptions of success, and 
reported outcomes) related to three variables that describe the quality of 
the dyad relationships among members. These include how an organiza- 
tion perceived other members' i) overall value an organization brings to 
the collaborative (index score of power and influence, level of involve- 
ment, and resource contribution); ii) trustworthiness of the organization 
(index score of reliability, mission congruence, openness to discussion); 
and iii) how often organizations interact with that member organization 
(measured as frequency of interaction on a likert scale). We examined 
independent variables that measured organizational type, perceptions of 
outcomes achieved, and reasons for success in the collaborative. We 
found some variables significantly related to the variation in our three 
outcome variables (Table 2). For organizations identifying increased 
services as one of their collaborative 's outcomes, a higher frequency of 
interaction was predicted. Organizations identifying lower frequency of 
interaction were businesses, dental, faith-based, funders, law enforce- 
ment, and public health. Higher trust and value scores were significant- 
ly related to organizations identifying exchange of knowledge informa- 
tion as a reason for success in the collaborative. Businesses, govern- 
ment, funders, hospitals, law enforcement, non-profits, primary care, and 
public health organizations reported lower trust scores for their partner 
organizations. Finally, funders, non-profits, public health, and profes- 
sional organizations reported significantly lower value scores for their 
fellow collaborative partners. 

Finally, we used traditional regression models at the organizational 
level to explore the second research question in more depth. 
Specifically, we looked at why some organizations are perceived as 
more valuable or trusted than others. Using a stepwise regression 
approach for resource contributions as our independent variables, we 
found that a few types of resources contributed by member organiza- 
tions were significantly related to trust and value. Those organizations 
providing specific health expertise ((3=0.250, P<0.001) and paid staff 
(|3=0.250, P<0.001) received higher trust scores and those providing 
in-kind resources (|3= -0.217, P<0.001) were associated with lower 
level of trust. Organizations providing paid staff (|3=0.287, P<0.001) 
were also more highly valued. 



Discussion 

This exploratory analysis of 11 public health collaboratives has provided 
some insight into the demographics, network characteristics, and factors 
related to quality of relationships among members of PHCs. These findings 
inform what we know about the structure of public health collaboratives and 
how some organizational characteristics are related to relationships 
between members of collaboratives. From these analyses, we have identi- 
fied the following summative points. 

Public health collaboratives are unique in structure, although patterns of 
characteristics are evident. Overall, we found a high degree of variation 
among these 11 public health collaborative structures. These results 
demonstrate the uniqueness among public health collaboratives in terms of 
who is a member, their frequency of interaction, and the role that each 
organization plays in the network. Specifically, we were not able to identify 
a common role that public health agencies play in each collaborative. Some 
are central to the network, others are evenly embedded throughout the net- 
work, and still some find a role on the periphery of the network. Time (not 



Table 2. HLM results of outcomes, success, reasons for success 
and organization type on frequency of interaction, trust, and 
value (significant results only reported). 
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Coefficients were significant (P<0.05). Tfiere was only minimal model change between the unconditional 
model and the model with predictor variables; this means there was very little model improvement and there- 
fore, the predictor variables do not explain our understanding of the outcome variables to a large degree. 



measured) may play a role in these findings. It is not uncommon for a pub- 
lic health department to play a central role early in a coUaborative's forma- 
tion, often as a facilitator/ funder/convener, but as the collaborative 
matures, this role becomes shared by other members.^"^ 

The findings that transitivity and reciprocity mimic one another in a col- 
laborative indicate that better information flow can occur when members 
form reciprocal relationships. It is not uncommon that a small set of mem- 
bers take the responsibility for attending meeUngs, leading subcommittees, 
and working outside meefing Umes. When more members participate in 
these kinds of activifies, greater opportunities for reciprocity emerges. 
Increased reciprocal exchanges in turn will lead to more closed triples (any 
three members and the relationships between them - the measure of tran- 
sitivity) and in turn better overall informaUon flow throughout the collabo- 
ratives. Differences among organization types in the network are a key fac- 
tor in how these partners value and trust one another The many types of 
organizations that participate in public health collaboratives bring both 
benefits and a degree of complexity to these collaboratives that make work- 
ing together a challenge. Mays and Scutchfield,^^ refer to this characteris- 
tic as breadth and link this to an opportunity for more and diverse 
resources. What we learned from this analysis is that different types of orga- 
nizational partners perceive the trust and value of their partners in unique 
ways. Funders are perceived as the most valuable in this data set. This is 
not surprising given that funding is a key factor in the iniUation and sus- 
tainability of public health coUaboratives.^^'^^ While the difference was not 
significant, it was surprising that public health agencies were not perceived 
as one of the top three valued organizations, given their prominent and 
trusted role in public health collaboratives. However, while other organiza- 
tions might be perceived for their value more frequently, public health is 
ranked as the most trusted. This is a sign of the important role that public 
health agencies play in facilitation roles^^ and explain why public health 
agencies are often identified by members as key players in these types of 
collaboratives.^"^ Differences in perceptions of partners' value and trust of 
one another can inform practice, specifically when collaboratives are iden- 
tifying potential roles and funcUons of members. While it may be surprising 
that some organizations can be highly valued but not trusted (or vice- 
versa), it could simply mean that organizations play different roles in these 
collaboratives and it may not be necessary that they play both of these roles 
simultaneously The implications of this guide public health collaboratives 
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in strategic thinking about how to leverage organizations with both l^inds 
of characteristics (for example, identifying organizations that are trusted 
but not perceived as having as much value to fill roles where trust is a key 
factor for success; or for example, identifying valuable members such as 
politicians and funders who do not regularly attend meetings (something 
that often breeds trust) but play important roles in the work of the collabo- 
rative (such as making policy change or implementation of programmes). 
Network members' characteristics play an important role in social 
exchanges. Certain resource contributions, a network member's perception 
of outcomes achieved, and identified reasons for success play a role in the 
quality of partner interactions (the frequency of interaction and levels of 
trust and value among members). First, a network member's resource con- 
tributions seem to influence how they are valued and trusted. The amount 
and type of resource that members contribute appears to be a factor in how 
much members trust one another. Organizations providing paid staff and 
specific health expertise were more trusted in these public health collabo- 
ratives, while organizations providing in-kind services received lower trust 
rankings. It is questionable whether organizations increase their trust lev- 
els when they commit paid staff and expertise, or that organizations that 
have more trust with other members are willing to commit these kinds of 
expertise. In other words, are members willing to commit more than in- 
kind resources when greater levels of trust exist in the collaborative? In 
these data, only increased services was associated with frequency of inter- 
action. Other outcomes reported by the respondents were not associated 
with frequency of interaction. This finding is counter to the process that is 
most common within public health coUaboratives; that is, that meeting reg- 
ularly with more partners will result in more outcomes.^^ The lack of evi- 
dence that meeting more often leads to better outcomes informs future 
research questions to determine how coUaboratives can achieve outcomes 
and demonstrate success in ways other than adding new members, meet- 
ing more frequently, and demonstrating more activity. A focus on demon- 
strating the quality of interactions among members is the next step in 
understanding how working together in these processes can lead to more 
and better outcomes. 



Conclusions 

One of the difficult aspects of understanding factors to effective collabo- 
ration is the variance among the missions, process, and outcomes of each 
collaborative. While it would be ideal to identify a specific set of factors that 
will determine when a collaborative will be successful, we found in this 
study that the unique structure of each collaborative makes this a challeng- 
ing exercise. The implications of these findings indicate that the research 
linking collaboration to outcomes must take into consideration the complex 
system of factors that influence outcomes. This includes the complex orga- 
nizational and governance structures, variations in goals and processes, 
and the challenge of measuring outcomes that can be attributed to collabo- 
rative activities. Evaluators and researchers should use caution when gen- 
eralizing about what an ideal collaborative structure looks like and why 
those factors might lead to successful outcomes, given the complex nature 
of the nested features of the inputs and outputs that can be considered. The 
practical implications for taking a systems approach to studying collabora- 
tion are evident. While getting a sense of factors that lead to success will 
inform collaborative practices, it is as important that each collaborative 
articulate its goals, take account of whether the process they are engaged 
in gets them closer to that goal, and then develop action steps that help 
them meet those goals.^^ To engage in this, a quality improvement process 
should be implemented that includes planning (setting goals), implement- 
ing collaborative activities, collecting data on the collaborative process, and 
analyzing whether the process is meeting the goals. Future work on the 
evaluation and analysis of coUaboratives should take into account differ- 
ences among coUaboratives, specifically the community context {e.g. geog- 



raphy types of collaborative leadership, funding, governance structures, 
missions, etc.) as a way to identify factors related to effective collaboration. 

This relatively smaU, exploratory study of public health coUaboratives is a 
modest start to understanding the broad and largely unchartered landscape 
of public health and community collaboration. From this study, we have 
articulated a number of future research questions to contribute to our 
understanding of how working together collaboratively can lead to more and 
better outcomes. First, further study of how different sectors involved in 
community coUaboratives and their variability in perception of the goals, 
governance structures, and resource availability (defined as breadth by 
Mays and Scutchfield^^) can provide guidance on how to manage the diver- 
sity that comes with bringing together organizations from many sectors. 
Second, these data showed some evidence that there exists a large degree 
of variability in member's perceptions of whether their coUaboratives have 
been successful and which factors contribute to success. While we expect- 
ed to find more consensus within each collaborative of the perceptions of 
success, we consistently found more disagreement among respondents 
than agreement. In other words, members of a collaborative vary widely in 
how they report their perceptions of whether the collaborative has achieved 
success or not. This leads us to question how members of public health col- 
laboratives are defining success. It appears now that definitions of success 
include stark differences and a lack of consensus by members regarding 
whether the work of the collaborative has achieved that success. An impor- 
tant question that needs further examination is How can coUaboratives 
measure their success if perceptions of goal attainment vary so greatly among 
members? The data from this analysis indicates that exchanging informa- 
tion and knowledge, sharing resources, bringing together diverse stake- 
holders and informal relationships created were among those reasons most 
frequently reported as factors associated with successful outcomes, giving 
us some indication of what leads to success. On the other hand, respon- 
dents who reported more outcomes also reported lower levels of success, 
indicating that perhaps having too many or disparate goals could have an 
impact on perceived effectiveness of the work. These data represent a small 
sample of coUaboratives selected from the PARNTER dataset. Since this 
sample was pulled, the PARTNER dataset has grown to include over one 
hundred similar datasets. Future research includes analysis to answer 
these and other research questions with a dataset containing a larger num- 
ber of PHCs. This study allowed us to identify future models to analyze as 
we explore ways for public health departments to improve their work toward 
strengthening their network of partners in order to address population 
health outcomes. 
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